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This report should be read in conjunction with last year's.* This is 
particularly important to obtain a routed vfew of our work on vi- 
sion. In fact, much of what we say this year is logically prior to 
much that we said last year* Thus last year we discussed the ab- 
stract and theoretical problems related to the interpretation of pic¬ 
tures presented in a clean form (as line drawings or as subsets of 
an abstract retina) while this year wo say more about how to obtain 
such pictures irom the real world. The reason of course lies in 
the fact that the "higher level" work did not depend as heavily on 
prior solutions oi hardware and systems problems. 

We have not reported new work that lies directly In the line of de¬ 
velopment of ideas discussed last year* hi particular, we have deep¬ 
ened and generalised some of the theorems on computational geom¬ 
etry. But the new state of knowledge is indistinguishable from the 
old gn the level of discussion of this kind of report* Similarly we 
have not reported very new work which is still at a top primitive 
level of development to be presented intelligibly. This includes work 
on natural language processing, concept information, teaching and a 
number of mathematical topics. The time constant for a project cf 
this sort is longer than a year. These matters will be dealt with 
iu a further report due in the Fall of 1970* 

Analysis of Visual Scenes: The Concept of Vertical Problem-S alying 

One can use vision in many ways to find out about objects in space. 
Let us begin with, some of the simplest ideas, and then move on to 
Lbe deeper problems. If the camera can move, one could, use stereo¬ 
scopic correlations of two pictures. Or one might use local opera¬ 
tions to measure motion parallax* These do not solve all problems. 
Stereoscopy does cot work well on featureless surfaces or curved 
boundaries; motion effects are misleading on plain or shiny objects* 
Nevertheless, both methods can help find the three-dimensional loca¬ 
tions of parts of visual objects. 

Focusing provides a third method of location. Its distinctive feature 
Is that it needs only a single eye in a fixed location, provided that 
the eye has a wide enough optic aperture* For then the Idea of 
measuring distance tty stereoscopic parallax merges with that of 
measuring distance by finding the best focus setting. Figure 1 shows 
two views of an object lying on a plane; in the photograph it is 
hard to sec the object because of camouflage. (The machine sees 
from a little below the viewpoint of the left picture.) Berthold K. P. 
Horn has developed a program that can find the Lens setting for best 
focus at each point in the visual field, The procedure, applied to a 
Series oi points along a horizontal scan through the middle of the 


See p. 43* 








cube, yields the profile shown in Fig. 2. fThe location of the back¬ 
ground is less definite then that of the cube because of the back- 
ground'a obliquity to the camera.) Horn’s program uses local Fou¬ 
rier transforms and compares the relative energy in the high and 
low spatial frequencies, [r servo-controls the lens to maximize the 
highs, and it focuses at least as well as one can do manually. 



Fig, I 



Horn will discuss the focusing procedure in detail in his thesis 
* Shape from Shading: A Method for Finding the Shape of a Smooth, 
Opaque Object from One View". Two techniques are available; one 



uses a fast Circular scan to obtain a periodic function characteristic 
ol a large neighborhood; the other makes a faster scan of a smaller 
square. An Operation manual (Horn, " Focusing*, AJ, Memo 16-0) gives 
instructions for using the system with the PDP-6 computer and its 
optical-mechanical accessories, as well as some ol the theory ot 
the focusing proredure. 

Now we have described three methods for optical range-finding;. 

There arc many other ways to attack just this one aspect of visual 
technology — such as flying-spot, scanning from a site off-center 
from the camera, holographic methods, and even optical radar. We 
must not, however, allow the myriad ol technological possibilities 
to divert un from the deeper — and incompletely understood *- prob¬ 
lem of developing a visual system flexible enough to deal with real- 
world problems. The goal we have set ourselves is to find how to 
make a system that can approach the versatility of human vision. 

One outstanding feature of that system is its "pass Lyeoe$S * --the 
great extent to which it can see without much special preparation 
or interaction with the objects in the scent. The secret lies in the 
intelligent viewer's ability to combine what he sees with what he 
knows about hie world. 

To pursue this, we have concentrated on the problem of reconstruct¬ 
ing a three-dimensional structure using only a single monocular pic¬ 
ture. People are quite good at understanding what is shown in a 
photograph; we should like to know how to make a machine do this. 
Now, with a very few exceptions, the many past attempts at com¬ 
puter analysis of scants have been rather fruitless, and we should 
try to understand what went wrong. Almost all of those past at¬ 
tempts followed the same general plan: the picture is subjected to 
a sequence of transformations; each transformation is intended, in 
turn, to produce a successively more abstract representation until, 
finally, one obtains the desired description of the scene* Typically, 
such a sequence might he: 

1) Remove noise (by clipping, smoothing, etc 4; 

2) Enhance features (by boosting gradients, etc.); 

3) Extract features (finding edges, vertices, etc,); 

4) Group features into objects {by regions, parallelisms, etc.); 

5) identify objects (by partial matches, etc,). 

Although there is a great deal of plausibility to this idea of pro^ 
grossing relentlessly from local to global, the concept of serial 
stages of pre-processing does not actually work well In practice. 

It is simply not suited to the real problem. Errors and assumptions 
made at each level are passed on to the next, and even if each 
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is quite clover at hgw it handles its data, the accumulation of mis¬ 
takes over many stages leads to chaotic over-all results, Thu basic 
grammar of the problem is too context-dependent. The appearance of 
an object's features (and even their occurrence or non-occurrence) 
usually depends on global aspects of the arrangement and illumina¬ 
tion oi the scene. One must cope, for example, with 

1) Direct line-of-sight occlusion of parts of objects, 

2) Shadow occlusions that depend on the directions of lighting, 

3} Highlights, 

4) Reflections, 

5) Textures, 

6) Decorations, 

7} Many other interactions between visual features and 

spatial forma. 

Accordingly, the inevitable ambiguity problems met at each level -- 
"la this nan edge or not?' or ’’Are these two features part oi the 
same object?* — are often not solvable at that level. One can se¬ 
lect a plausible interpretation only by using a wider vartety of 
knowledge about the real world — knowledge that ranges from prin¬ 
ciples of optics a::d geometry to knowledge about, the particular en¬ 
vironment and the objects likely to be In it + 

In the traditional processing sequence outlined above -- we shall 
call It the horizonta 1 vision system — different kinds of knowledge 
are implicit at each level. The principles oi optics are involved 
because each visual point represents a distribution function oi space 
points in a way that depends on focus, scattering, reflection and 
noise, rn the extraction of features, the processor must know wheth¬ 
er the objects are likely to have straight edges, or texture boun¬ 
daries, or polished surfaces. In analyzing a room, to give an ex¬ 
treme but real example, imagine the resulting chaos if the system 
did not know the significance of a picture frame f 

At the level of grouping features and identiiying spatial bodies, we 
have already seen (A.I* Group, P. R. V; page 43) how problems of 
projections and occlusion of three-dimensional objects lead us 
away from the simple template-matching schemes that work fairly 
well for two-dimejisional problems. We found, however, that many 
of these difficulties could be handled by symbolic-description sys¬ 
tems, and we snail assume that the reader is familiar with Prof. 
Adolfo Gunman's work, either through the survey in Progress Re¬ 
port V, or through his Doctoral dissertation. We now conclude that 
the lower-level aspects of vision, too, are best treated as problems 


Lti artificial intelligence to be handled by a mixture of general meth- 5 

oda and special knowledge. Because the different kinds of knowledge 
interact at different levels, we must provide channels tor such inter¬ 
actions so that hypotheses about high-level things like objects -- 
perhaps proposed by heuristics that use local evidence -- can be 
eonfirmedj rejected or revised by returning to other levels for other 
kinds of evidence. We use the term vertical system for tills type of 
organization. 

We have not yet enough experience with vertically to discuss it ab¬ 
stractly. But we now know a substantial amount about its application 
to visual problems; the body of this section reports what we have 
found so far. 

Optical Anatomy of a Sim ple Scene 

In Fig. 3 we see three cubes with dull white painted surfaces against 
a dark background, illuminated by concentrated light from a lamp 



Fig. 3 



Fig. 4 
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above and to the fight of the camera. The data and methods used 
here arc from work by Arnold K, Griffith. Figure 4 shows three 
plots of the- light Intensity;, measured along three horizontal scans, 
each of 1000 points across the picture. 

It is difficult to discern very much in these plots. The next illus¬ 
tration., Fig, 5, shows the result of applying, to a sequence of such 
cross sections, a kind of second-derivative operation averaged over 
enough acjjacent sample points to give fiat plots over regions tliat 
have reasonably uniform gradients, (The photograph and the meas¬ 
urements were taken from slightly different positions,) 



We have marked a number of typical features; 

a is an outer edge. Against the dark background, it Is a sim¬ 
ple, sudden change in intensity, giving a typical ”bipolar" 
pulse in the smoothed second derivative. 

b is a more symmetrical feature; it is due to the '’highlight 1 ' 
reflection on the front edge of the lower right cube. 

b r is a superposition of the effects of a b and a small a. 
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c_ la the reflection of the bright surface of the upper cube in 
the top of the lower right cube* Although the paint there la 
dull, this surface is seen at a low angle, and this enhances 
reflections* 

d la the crack between the lower cubes * The brightness 
measurements are all logarithmic, and truncated so that the 
plots won't overlap. 

c is the spot of dirt on the upper front corner of the lower 
right cube. The edges and corners of objects often have high¬ 
lights and often are dirty. 

f is a shadow boundary visible on the dark background. 

r is another shadow boundary,, somewhat less sliarp because 
of penumbra and depth of focus. 

The internal edges, such aa at b, of uniformly colored objects often 
give small signals. Their width, In otir plots , is an, illusion due to 
the smoothing operation; on sharp corners* the highlight line is 
very narrow and easily missed by a coarse scan. 

The history of attempts to write edge-finding and edge-following 
programs is long and inconclusive. Because the results were so 
obscure, Annette Herskovits <AI Memo 103) and Griffith made sepa¬ 
rate studies of the edges of geometrical objects; both concluded 
tliat the most common phenomeiia were superpositions of three ef¬ 
fects (see Fig* 6)* 


(1) a Simple Step 




Fig. 6 









& They both investigated various detection filter methods for these. 

Griffith's thesis will include a theory of optical defection of edges 
under various assumptions about their Intrinsic character and about 
, fli^ kinds of noise one might expect in a vision system. The most 
sensitive methods for detecting edges use two-dimensional operations, 
but these are very expensive with conventional hardware* because of 
the large amount of high-resolution information. 

When a compromise must be made, it is not especially good simply 
to use a coarser homogeneous scan; for instead of the arrangement, 
of Fig. 7, one can use that of Fig. 3, which has the same mean den¬ 
sity but is better at catching thin edges. 


Ftp. T Fig. 8 

Now we apply this idea to some more realistic, cluttered scenes. 

We mark with short dashes the local maxima of the output of the 
edge-filter, along a mesh of fine horizontal and vertical scans. The 
problem remains to convert this set of local features into lines, and 
then into objects. Figures it and 10 show the results of a process that 
uses a projection operation sensitive only to straight-line segments. 



Fig. 9 















Although it. is not good for closo-packed features, it is conservative 
and does not propose many [also lines* Griffith^ thesis will give 
details. 
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Fig, 10 


Another kind of process works in a complementary manner; it labels 
points whose neighborhoods are relatively homogeneous, Then the 
system finds the boundaries ol the connected regions of -such points, 
Thomas 0, Binford (AI Memo l&Z.) describes experiments on such a 
system. There are many problems in deciding how? to reduce the 
region boundaries to useful line-descriptions. We see fn Fig, 13 the 
result of such a system applied to the relatively simple scene shown 
in Fig. 11. 



Fig. 12 
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Still another approach to line-finding uses a two-dimensional local- 
gradient detector, followed by a scheme for assigning the locally 
maximal features to edges, as in the early work of L. G. Roberts 
(Ref, i) + Richard D, Greenblatt is developing a system of this sort. 

Problems exist at every stage of such processes, At each level, 
the selection of relevant features requires some a priori knowledge 
about the local world. Our vertically thesis holds that one cannot 
expect any one decision policy to work over a very wide range of 
situations, but that even a little feedback in this selection wilt help 
considerably. For example, each of the systems mentioned above 
will miss some edges of some objects, because of the problems of 
resolution, illumination, focus, contrast, texture or noise. If the 
system misses an interior edge, the SEE program (or rather, one 
version of It) may have to propose one object in place of two, as 
Ln Fig, 13, or two objects in place of one. 



Fig. 13 


Assuming we are in a world of geometrical bodies, there is a varie¬ 
ty of ways in which to use knowledge of the fact to propose correc¬ 
tions, (These will have to be verified, but proposing them is most 
of the battle J 



Fig. 14 















II 


Missed edgt'S t tar example, arc often related to concavity (see 
Fig. I4) f and, in a rather Bayesian way, this suggests a search tar 
missed lines radiating from the- concave vertices into (he figure 1 s 
interior. 



In Fig. 15* wo indicate proposed lines of several kinds; V directly 
to other vertices; E interior extensions of the vertex's edges; and 
L an (absolute) vertical edge (very common in real interior scenes). 
One might further propose parallels and linos that make coolocal 
triplets. 

It is remarkable how much can be done with such a lino proposer, 
hi the ease of structures made entirely of rectangular solids f Prof, 
Manuel Blum showed (Fig. 16) that a remarkable number of interior 
edges can be reconstructed just horn the outer profile of the scene. 




T lg. £6 
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The number gf lines proposed by such a scheme can be held to the 
order of tens, rather than of thousands. And the cost of verifying the 
existence of an edge with a specified Ideation in enormously smaller 
than that of finding all such features independently, because -- for 
the same statistical confidence -- rejecting a particular null-hypothe¬ 
sis is always much easier than screening all of a large family of 
possibilities. The use of proposer-verifier system can thus reduce 
the total picture-processing effort by relaxing the tolerances on the 
early* brute-force, feature-finding stages. In Ids forthcoming thesis* 
Griffith will give details of a complete system* already working, 
that does this. A first stage finds some of the edges in the scene. 
Than new lines are proponed on bases of parallelisms , region com¬ 
pletion, etc*, and verified try the detectors mentioned earlier,, 

By using a priori information, one can often get much more out of 
a picture than might seem to be in it* Assuming (correctly) that 
the sphere in Fig. 17(a) is uniformly colored and that the light 
cornea from a compact source, one of Horn's programs is able to 
reconstruct the surface by solving the appropriate differential equa¬ 
tions. Then this program produces the stereoscopic pair of Fig. 17(b) 
and 17(c), (Some readers will be able to fuse these by looking at a 
virtual point beyond the page.) The sharp shadow detail confirms* 
with the picture, the hypothesis of sharp illumination* Horn will pre¬ 
sent details of this program In his thesis. The method is quite com¬ 
plementary to stereoscopy since it relies on uniformity of surface* 
whereas stereoscopy and focusing need. Inhomogeneous surface detail 
or discontinuities. 



Fig, 17(a) Fig. 17(b) Fig. 17(c) 


Even if the surface is not of uniform color and texture, there is still 
much more that can be done with a manocular picture. Lawrence JF. 
Krakauer in developing a system that analyzes scenes such as a 
bowl of fruit* The process be gins by locating and analyzing illumina¬ 
tion maxima; it appears that, from their intensity-shape behavior 
one can* In many cases, distinguish dull from shiny surfaces. Local 
maxima connected by an illuminated band are likely to be on the 
same object, and Krakauer is testing some other heuristics for as¬ 
sociating highlights with fidget (Fig. IB). 






Digitized image of 
Some Fruit 


Edges and Maxima, 
Enhanced 

Fig* L& 


Features 

Alone 


Krakauer’s goal is to discover visual charade rlatica of surface 
properties,, and to establish heuristics tor grouping "non-geometric 11 
features into natural objects, perhaps as Guzman did (or geometri¬ 
cal objects. 

The Vision System 

We have discussed Gusman's SEE program {see page 43J; this 
program assumes a scene description in terms oi edges, verti¬ 
ces and regioais, and produces a proposed assignment of these .fea¬ 
tures to a set of three-dimensional bodies. Guzman’s thesis describes 
in detail a more advanced version of that project. It Includes addi¬ 
tional heuristics for linking parts oi objects* analysis of the system 1 s 
behavior* discussion of and heuristics lor correction of mistakes be¬ 
cause ol pre-processor errors* and some analysis ol the system er¬ 
rors due to inherent ambiguities in ordinary scenes and In a variety 
of standard "optical illusion’ 1 scenes. Guzman's thesis also includes 
some observations about the problem of matching features between 
stereo pairs. In particular* the following simple technique, when It 
is embedded in a vertical system, will solve the majority of such 
problems. Consider two views of a geometrical scene {Fig* 19). In 
any stereo pair, one can dissect the two pictures into sets of match¬ 
ing line-pairs, defined by the planes through the two eye-points, Any 
physical object visible to both eyes will be sandwiched between the 
same highest and lowest such lines, as suggested by Fig, £0. For 
two different objects, it is unlikely this will be true by coincidence, 
especially If the objects have more than one visible face, since the 
sandwich proposition is true also for each facel Thus, once enough 
point features are identified in the monocular pictures, one will have 
little difficulty in matching them between the pictures, and expensive 
cross-correlations should be unnecessary* This matching works well 
even when the two viewpoints are far apart* 
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Fig. 20 


In another attack cm stereoscopic vision* David N, Perking, Jr. is 
developing a system that compares two views and produces depth in¬ 
formation. His matching procedure is complicated by the requirement 
tiiat it be tolerant of the many kinds of errors that pre-processors 
are likely to make. It proceeds by matching vertices and arcs topo¬ 
logically, with some geometric constraints* and with a back-up and 
search procedure for fifating maximal matches of substructures when 
tiicre arc possible local ambiguities. Then* given the best topological 
match, the program proceeds to analyse the arcs that axe thus pro¬ 
posed for matching to obtain space curves,, For example, the stereo 
pair of Fig, 21 is converted, by Binford’s TOPOLOGIST system, to 
the views ol Fig. 22(a) and 22(b), 

Then Perkins's system, on request, produces views as seen from 
the right side and from above in Fig, 22{c) and 22(d) (shown here 
without suppression of lines that might be hidden by surfaces), 
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Fig, ZI 



(a) (b) 




Fig. %Z 









We now have a complete horizontal system of programs connecting 
Binford r s topological pre-processor with Co-Email's SEE program 
and on. through to Patrick Winston's new system (described later in 
the report) that recognizes some particular typos of objects -- e.g., 
wedges, pyramids, and rectangular blocks — and learns to identify 
some multi-object structure such as rows, towers and bridges. 

Prolessor Ho&akerc N, V. MahabaLa developed die TDPOLOGKfT-to- 
SEE interface. This program, SETUP ("Pro-processor lor Programs 
Which Recognize Scenes”, A.L Memo 177J, gobbles a list of line seg¬ 
ments and prodncee a complete topological description ol the graph 
formed by the lines. Because «uch features as lines and vortices 
found by pre-processors have some uncertainty in location, there 
arc usually serious problems in deciding when two edges are really 
the same, or In connecting edges and localizing vertices. SETUP 
contains heuristics for plausible guesses about such matters. All 
lines are treated as enclosed within strips of a certain width, and 
all problems about closure, intersection, containment, coliifcearity, 
etc., are resolved by procedures that are based on a single predicate 
about the orientation of a point with respect to one ol these half-line 
strips. Although this may mpt be any better in performance than 
Other segment-*joining and line-grouping criteria, it is probably no 
worse, and Mahabala'e system is distinctive in having greater logi¬ 
cal clarity than any Other System we have seen before. It is there¬ 
fore much more likely to be improved by advances in theory! 

Finding the edges themselves is still a problem. When one knows, 
a priori. , that they are straight, the criteria Griffith and Herskoyits 
developed are probably adequate for practical purposes, and these 
are further subject to substantial heuristic speed-up innovations. 

For Lhe mor« general problem of describing an ordered set of 
points (such as one obtains as the boundary' of a ”homogeneous re¬ 
gion") as a curve, we need a systematic way' to apply various kinds 
of a priori knowledge. At present, we liavc a variety of such at¬ 
tempts, such as P0LY8EG (Griffith, A.L Memo 131), the Greenblatt- 
H olio way line-finder (A.L Memo 101), and three others, by Rinford, 
Greenblatt, and Jayant M. Shah. Unfortunately, none of these Is well 
enough understood to be considered theoretically firm. A variety ol 
otiier general-purpose curve-segmentation procedures has been de¬ 
scribed Ln the literature. But nO One has really come to grips with 
the basic problem of incorporating the relevant a priori information, 
and we conclude that this is one of the aspects to be faced in de¬ 
signing the vertical vis inn System, 

Richard Orban has developed a new program, ERASER, which de¬ 
tects and removes shadow boundaries from geometrical scenes. 
ERASER te sembJ r.h ERE in that it uses the same class if icatioji of 
vertex features, but it also usc3 information about the relative 





brightness of regions. Its heuristics are based largely on the abun¬ 
dance of L, T and X types of vertices On the boundaries of shadow 
regions. ERASER is designed to criticize the output ol SETUP, to 
remove shadow boundaries, and, then to re-submit the result to 
SETUP before passing the problem on to SEE. It will not remove 
all shadows, but it is conservative about not removing real edges. 
Examples of ERASER'S performing well are given in Fig. 23. 






The ERASER system also contains some heuristics that Hinford de¬ 
veloped for guessing the direction ol illumination; this is used to 
bias the operation of ERASER and is available to the rest of the 
system. In geometrical scenes* the system can measure the angles 
between real vertical edges and those shadow boundaries that appear 
to lie on horizontal surfaces in order to get a quantitative measure 
of illumination angle. As an application of vertically* the operation 
of both ERASER and SEE could be enhanced by verifying^ at a high¬ 
er level, that some of the shadow boundaries Ue across otherwise 
uniform surfaces (say, by using Perkins's stereo system) or by 
verifying, at a lower level* tlial the proposed Inner shadow bound¬ 
aries are less than sharp, i.e., have penumbras. 
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Me c hanleal Structure Analysis of Visual Sc ene * 

Blum and Griffith have written a program that can analyze the sta¬ 
bile Ly of the threEf-dimensig.mil structure of rectangular blocks- The 
program uses this analysis in a planning scheme to propose the 
order in which the structure is to be built. Even In manipulating 
toy blocks, there are problems; one can ask which of these struc¬ 
tures can bo constructed with one hand. 

.Retorting to Fig. 24, tha one On the left cannot be built:, the pro¬ 
gram asserts, because there is nc. stable three-block sub-part oi 
the structure. But with another interpretation ot the rules, one could 
first assemble the upper three blocks on the floor, then lift them 
into position. We expect our more advanced construction-planning 
programs to be able to use more advanced strategies In which sub- 
assemblies are so identified. 



WLnSton is completing a program that learns to recognize types of 
structures from sequences of examples. We consider it to be a 
major advance in the areas generally known as concept-formation, 
or machine - learning . Given a scene (represented by a collection of 
regions, as produced at the output of the SEE program), WinStcWs 
program attempts to describe the scene in terms of elementary ob¬ 
jects and already-known sub-structures and relations, using a de¬ 
scriptive language reminiscent of that Dr. Thomas G, Evans used 
(Ref. 2), but Winston's language is further developEid, 

For example, the scene of Fig. 25 leads to a description like that 
shown to its right* We tell the program this is a picture of an 
AftCfl. 

Next, we inform it that the picture of Fig. 26 is not-an-ARCH. The 
program then proceeds to compile a new description (of ARCH), as 
shown to the figure's right. 

This new description oi ARCH is obtained by comparing the descrip¬ 
tions of the two scenes, thus providing a second-level description. 
The must-not-abut relation is the out standing difference it discovers, 
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and, it modifies the description of ARCH to require that the abutting 
relation not hold between the two supporting blocks. (Winston's pro¬ 
gram is equipped ab initio with heuristic a lor proposing support and 
non-supporting contact.) 




Fig. 25 




Fig. 2e 
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Now,, when we give the program the next figure (Fig + 2?) as another 
example of nOt-an-ARCH, it again modifies the ARCH description, 
this time requiring (rather than just mentioning) the supported-by 
relations. 



Fig, 27 





Fig. 2fl 
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Again, this change occurs because the change in support is the most 
prominent difference the comparison program found. Finally, we 
a how it one more example of an ARCH, and it recompiles a de¬ 
scription in which the requirement that the top-object he a rectangu¬ 
lar brief: has been removed (Fig. 23), 

There are many arbitrary elements in how this program decides 
what to do at each step -- which differences to give highest priori¬ 
ties, how to match up different description networks, what explana¬ 
tions or excuses should be assigned to the differences it notices. 
These commitments are kept on back-up trees., so that it is possible 
for the program to recover from at least some disasters* The goal 
is to obtain behavior that would be plausible in, if not typical of, a 
child. The particular concept that the program develops from a cer¬ 
tain sequence of examples will depend very much on those examples 
and on the order in which they are presented — as well as on the 
connection of concepts the program has already acquired at that 
time. The experimenter will not always get the results he wants or 
expects! We cannot expect the first real concept-learning programs 
to be foolproof, any more than a teacher can expect his favorite in¬ 
structional technique always to work; but here at last we have a 
clianee to understand precisely how a training sequence interacts 
with built-in and previously acquired ingredients of the system. 
Winston's methods are related to earlier ideas like those in Newell 
(Ref, 3) and emphasize the use of explicit descriptions. While there 
is some similarity, in strategy, to the Feigentaum-Simon EPAM 
scheme (Ref, 4)„ the result is pointed toward a true structural net¬ 
work description of the concept. It should therefore lend itself bet¬ 
ter to direct analyses of examples, instead of having to work 
through synthesis by generating and testing proposed examples. 

More generally, having a description (rather than just a test) for a 
concept seems absolutely crucial. The advantages include: 

1) The possibility of making deductions about the concept, 
including its consistency with a proposed detection scheme; 

2) Combining several descriptions ip non-trivial ways; 

3} Comparing and contrasting descriptions, as in Evans's 
program; 

4) Using the description to generate, rather than merely to 
select, what next to do in a search program. 

Similarly T in search processes, one could combine several descrip¬ 
tions to obtain Summaries of the information acquired in exploring 
different alternatives. In most earlier heuristic search schemes, 
the procedures usually have to abandon almost aR information ac¬ 
quired in the course of unsuccessful exploratory attempts because 
of inadequate descriptive facilities. 
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The A.L Group is now committed to a broad attack on the problems 
of symbolic learning* through the application of heterologlcal kinds 
Of knowledge to the analysis of descriptions. An essay (A.L Memo 
135) gives a preliminary statement of our plans for this project, 

TliCOru m- Prtjvi ng 

Gerald J, Sussman has implemented a theorem-proving' program that 
uses the Resolution principle with an assortment of retrieval meth" 
od3 and other heuristics to make deductions in predicate calculus. 
Although there has been a number of Interesting results, we never¬ 
theless believe that the use of this technique, as an approach to 
artificial intelligence, is receiving much undue attention today, and 
we do not plan to give it a large place in our future activity unless 
some new and impressive demonstration of its power comes to light. 
Suss man has been experimenting with a variety of means for com¬ 
bining the deductive strength of predleale-calculus resolution with 
heuristic flexibility of teS-n formally constrained problem"solving 
schema, hut his conclusions arc not encouraging. An example that 
puts one of the problems into a nutshell is this: suppose one is 
given 

A ==> B and 

C D, and 

(A B) & (C =?■ D) => E, 
and one wants to deduce the simple conclusion 

E. 

To be sure, the urogram manages eventually to obtain E, but only 
after many steps ol converting the given statements into expanded 
"normal* forms that are tn themselves rather meaningless. 

This would not be so bad in Itself, but it wouid become an acute 
problem if one were to try to give the theorem-prover additiojial 
advice about wliat to do in other situation, since the kinds of siting 
tions for which we can describe such advice are hard to represent 
in terms of the fractured internal expressions the Resolution sys¬ 
tem creates for tbs own use. In another experiment Sussman modi¬ 
fied: the representation of this problem so that the =S* symbols were 
not recognized by the prover ns meaning implies , and he added a 
separate set of axioms for using a more natural deduction method. 
Now the Resolution system produced a better proof in less time, 
even though it had to work indirectly through the new axioms! No 
doubt this particular problem could be ameliorated by some variant 
oi the many combinatorial schemes that are being widely studied to¬ 
day, hut we feel that, once such systems attempt to solve "real* 1 
problems, all such devices will fail as menu stop-gaps; they do not 
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help one Co come to grips with the construction of the kinds of cog- 
notive models we think are needed to solve hard problems by using 
accumulated knowledge and experience. Sussman's system takes 
some steps in this direction by maintaining its statements in a 
structure partially ordered by the substitution"instance relation. 

Nor is his system restricted to iirsl-order predicate calculus. But 
our gloomy expectations remain. Incidentally, we do not feel that 
comple teness, or even consistency, is of very large Importance. 
Logical completeness is more or Less inevitable in any system that 
knows a great deal, and consistency is not a notable feature In the 
intelligent machines that already exist. (The backers of the Resolu¬ 
tion method achieve the wrong kind of completeness; the kind ol 
completeness we need is for the problem-solver to be able to use 
any * natural* technique.) 

At perhaps another extreme, Carl E. Hewitt is developing a language 
for constructing deductive systems with the utmost heuristic flexi¬ 
bility. His language, PL ANN EH, is designed to allow use of knowl¬ 
edge and types ol representations of great variability. PLANNER 
must be one of the least procedural languagesto a large extent, 
one cam specify what one wants done rather than how to do it. Con¬ 
sider, for example, a statement of the form "A implies 0", As it 
stands, it is a simple declarative statement. But in PLANNER it 
can instead be interpreted as the imperative’ "set up a procedure 
that will see if A is ever asserted, and if this happens assert B 
also," Or, it can be interpreted; "set up a procedure that will see 
if B is ever desired as a goal, and if so assert A as a new sub- 
goal to be deduced." This is only the skeleton of the idea; PLANNER 
contains machinery for easy manipulation of many different roles of 
declaratives, imperatives, goals and deductions* in attempting a par¬ 
ticular deduction, for example, programs can specify suggestions for 
other theorems that should be used (and even in what order) to 
make the deduction. All statements are expressed in a powerful new 
pattern-matching language, MATCHLESS, in which control of proce¬ 
dures is specified in terms of the forms of assertions, rather than 
in terms of particular assertions. 

Because problem-solving often requires building up elaborate tempo¬ 
rary structures, PLANNER has machinery for handling statements 
that were once true In a model which may no longer be true after 
actions have been performed, and for drawling conclusions that may 
have to be deduced from such a change. Control of such matters 
is specified in terms of local states, to which are bound informa¬ 
tion about changes in. the data haae -- erasures, assertions, new 
definitions, etc. -- since the data were last updated. 

Hewitt describes his system qualitatively in a conference paper, 
and in detail in A.L Memo 160, He has now implemented! 



the language and is programming a compiler to obtain sufficient 
speed to permit full-scale experiment®. A£ present, he i® usmg it 
to study deductions about the manipulations of objects by a robot, 
as in "pick up all pairs oi cubes that are the same color, with one 
cube on top of a third cube that is in front of the other." Terry A, 
Winograd is completing a system that will train late such natural 
English statements into PLANNER assertions and theorems. 

Natura l Language Systems 

Winograd is completing a system for handling problems of computer 
understanding of natural language. The system is designed to accept 
information In normal English sentences, to answer questions, and 
to execute commands, using semantic information context to under¬ 
stand pronoun references and to disambiguate grammatically com- 
plicated texts. To do this, Winograd uses a new linguistic scheme, 
based, partly on the systematic grammar described in HalUday (1067 
Ref, 5} and in Winograd fl&60, Ref, 6) and partly on a special repre- 
sentation for both the grammar and the semantics. In previous at¬ 
tacks cm such problems, some worker® have used heuristic tricks — 
key words or matching of phrase fragments — or they have used 
non-heuristic formal grammars to do a detailed analysis of the sen¬ 
tence to which the semantics are to be applied, Wuiograd's system 
is based on a heuristic grammar that use® contextual Information in 
analyzing the sentence, carrying out the semantic analysis concurrent¬ 
ly; this is made possible by representing the grammar a® a program 
instead of as a set of static rules. 

Definitions of words, as well as the system"s knowledge about things, 
arc also stored as programs and are available to a deductive part 
of the system, This allows much more flexibility than one gets natu¬ 
rally from network® or rule-lists. The program can make long, 
complex, deductions fn answering question® or in ab®orbing new in¬ 
formation. 

The grammatical part of the system j® operating, with a quite com¬ 
prehensive English grammar (for a description, ®ee Winograd, "PRO¬ 
GRAM MAR, A Language for Writing Grammars", A.I. Memo 191). 

The semantic programs are still in preparation, to he combined with 
the deductive system, which will use Hewitt's PLANNER language. 

The entire system could be used for a general question-answering 
facility for any corpus oi knowledge programmed into the deductive 
system. The first applications will probably be concerned with in- 
stiueLing a robot and. with analysfs of children's stories at. the first- 
grade level®. 

Loosely Stated Mathematical Probl ems 

Several years ago, Daniel G. Bo brow completed a program that wa® 
able to solve some algebra problems stated in ordinary English. The 





mathematical material was rather sharply restricted to converting 
the sentences into simultaneous linear equations. Recently, Eugene 
Charniak has completed a new program that can solve some prob¬ 
lems like this, stated in Imprecise English: 

A train, starting at 11:00 a.m„ travels east at 15 miles per 
hour while another, starting at- noon from the same point, 
travels south at 60 miles per hour. How fast are they sepa¬ 
rating at 3:00 p.m.? 

Solution of this problem requires a number of intellectual skills. 
Among these are, of course, the ability to manipulate the English text 
to derive a well-formulated symbolic problem, and knowteclge of ele¬ 
mentary calculus necessary to solve the symbolic problem. Perhaps 
less obvious, hut equally important, is access to miscellaneous knowl¬ 
edge about the real world: for example, the knowledge that east and 
south are orthogonal directions, and enough knowledge about time to 
deduce that 3:00 p.tm is four hours later than il;Q0 a.m. 



JUOfrTOifcH 



CharniaJs T s system, CARPS fCAlculus Rate Problem Solver) (MAC 
TR-51}, translates this problem into an internal representation of 
which a general impression can be gleaned from Fig. 39. Tills struc¬ 
ture Indicates two objects, TRAIN and ANOTHER. Associated with 
each is a velocity given as direction and magnitude, a TIME that 
has a starting value for each train and a WHEN- CONDITION. The 
latter refers to the fact that moat calculus "rate problems* ask 
questions roughly of the form, "What is A when B? B ; in our base, 
we obtain, "What is the speed at which the trains are separating 
when the time is 3-00 p,m + ‘>" 

CARPS uses this information structure to formulate an algebraic 
manipulation problem, ROW FAST and SEPARATING indicate that 
what is desired is the rate of change of the distance between the 
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[■wo objects* The distance is the hypotenuse of a right triangle with 
le^s toward EAST and SOUTH. The lengths of the logs are obtained 
by multiplying the rate of travel by the time. The expression for 
the derivative is therefore 

a/ <4&[t-Tl 3) g + (60 [t-TE])2 j 

The desired value is at 3:00 p.m, (our WHEN-CONDITION}. The 
derivative is found by algebraic manipulation* and the program then 
types 

THE ANSWER. IS 72*246 M1LES/HOUR. 

For its algebraic operations, CARPS uses parts of the MATH LAB 
programs described elsewhere in this report. CARPS has been used 
to solve other problems taken verbatim from calculus textbooks. 
These problems deal with cones, spheres and shadows as well as 
distances* In each case, the program was given sufficient knowledge 

to parse the sciences, sot up an internal structure, and generate 
the equations. 

We do not wish to imply that the program is very strong at solving 
calculus problems. Frequently, a problem cannot be solved because 
the program is unable to handle the syntax used,, no method of solu¬ 
tion is known to the program, or the problem requires facts about 
the real world which the program does not know or could not handle. 

An example of the fast difficulty arises in; 

A (adder 20 feet long leans against a huusso. Find the rate 
at which the top of the ladder is moving downward if its foot 
is 12 feet from the house and moving away at the rate of 
2 feet per second, 

Must adults have little difficulty in visualising the situation described 
fn this problem* CARPS is stuck because it does not know in which 
direction the foot of the ladder is moving. The phrase MOVING AWAV 
m interpreted by people to mean "moving away along the horizontal 
ground on which the foot of the ladder is presumed to rest"'. CARPS 
however, does not realize that* 

Clearly, CARPS's lark of such real-world knowledge- cannot be cir¬ 
cumvented or ignored, A crucial pare of research towards problem- 
solving ability of this sort is concerned with the representation and 
use of such knowledge* 

We have discussed (A,I* Memo 105) preliminary studies attempting 
to analyze the sorts of knowledge used by a first-grade child in un¬ 
derstanding children's stories. Much of the calculus student's gen¬ 
eral knowledge" is already structured at the elementary-school level 
and we feel that progress in this area is essential tp the future 
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development of anything like "general intelligence''. We have decided 
to make this area a very large part of our work in the next few 
years. 

MATH I, A B 

MATHLAB is an interactive computer-program system that is being 
developed to facilitate creative work that involves extensive manipu¬ 
lation of symbolic algebraic expressions. MATHLAB is intended to 
be of lielp to research mathematicians and to applies of mathemat¬ 
ics engaged in "heavy manipulative work”* In the MATHLAB project, 
therefore, serious attention has been paid to human engineering and 
to efficiency and speed of operation as well as to basic mathematical 
problems and to the problem of programming the computer to exer¬ 
cise initiative and some judgment in selecting and carrying out trans¬ 
formations of algebraic expressions* 

MATHLAB is our best illustration of the idea of building "knowledge" 
into computer programs* The MATHLAB programs are actually quite 
capable in solving certain non-trivlal mathematical problems. The 
essential idea, however, is to include provisions for interaction with 
that knowledge so that the user can build and administer complex 
but we 11-understood procedures. At best, the user and the programs 
supplement and reinforce one another and march rapidly through 
symbolic-manipulation problems that would take the unaided mathe¬ 
matician countless hours. 

This last year, a new and much advanced MATHLAB was planned 
and partly implemented. Progress was made on faster parsing algo- 
rithms and on a representation of polynomials that speeds up addi¬ 
tion ajid multiplication. At the same time, a significant advance in 
programmed symbolic integration was achieved by implementing a 
decision procedure, due to Rlsch, for expressions involving rational 
functions, logarithms and exponentials. 

Alao this last year, an extension of MATHLAB to handle special 
functions defined as integrals has enabled Moses to verify or cor¬ 
rect some published tables of integrals, [n 105B, Iff* p. Maurer of 
the Argonne National Laboratory compiled a table of 150 integrals 
involving t]ie error function erffs)* Maurer was unable to verify, by 
hand, the following integral, which he included in the compilation 
with a note to that effect* In fact, it wa§ slightly incorrect as printed: 

y*2erf£a:K + J>]e p,), djL = 1erffax + b)e px (2 - 2p* + p 2 * 2 } 

p3 

4 pa x = pb + p?/2a - Zfl g -(ax+b) 2 +px 

p2 A 

b Wf(ax + b - £&)* 


„ P 4 /a - *p 3 b + 4ah ? p ? - Zap- * @a £ bp + &a 3 -e/ai£- 

--— e r ' '4 a 

4a V 






28 


Moses's prof ram was able to find the error * 

Here is an example of faxtortaation of a polynomial: 

(1) x**6 - 1 « O ftyped In} 

(2) x 6 - 1 = 0 (MATHLAB's response}. 

In line 1, the user typed an equation. (The syntax U awkward be¬ 
cause the typewriter is a one-dimensional device.) Lino 2 shows the 
computer’s response displayed in the conventional two-dimensional 
syntax of mathematics. 

(3) 'ff('WS) (input) 

(4) ■£jC- 1 J (5C^1 Kx^+X+l Hx^-X+l ) (response). 

in line $ r the usor ashed the system to factor the equation in line 2 
by requesting the operation PF (Polynomial Factorisation) to be ap¬ 
plied to tlie WS (Work Space). The Work Space is always the last 
expression known to the system. In the response given in line 1, 
the quadratics are not fully factored because cEie program is re¬ 
stricted to finding the smallest factors that, have Integer coefficients. 

Next we shall ask the system to integrate an expression^ differen¬ 
tiate the result,, and then check to see whether or not the result is 
identical with the original expression, 

l/(x**3 + A*x‘*2 + x ) (input) 

~ i (response) 

A* + + X 

■IsntegrateCws, x) (Input) 

IS TKE EXPRESSION 


(3) 

( 6 ) 

m 

(a) 


to be considered positive, negative or lero (response) 

(9) NEGATIVE (input). 

The computer wants to avoid terms in the integral which have com¬ 
plex values, if possible. This is the reason for the question posed in 
Mne &. Line 9 is the user's response, and line 10 is the integral. 
Obtaining the result involves use of a subsystem for handling partial 
fraction decompositions and one for integration of rational functions: 

(10) - 1/2 log{x e + fl« + 1} 


-A 


2-arcUn- 


tx * h . , i i 

-^5~T log(aO- 

qrt(-A ri +4) 


sqrtf-A^H) * 

We shall now differentiate the integral. 






( 11 ) 1 DERIY( "fJS „ x ) 


[input} 
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' —_ + i 

-A +4 

Wtll, the result is certainly not Identical with our original expres¬ 
sion (line &)* Tills is because the differentiation program differen¬ 
tiates a sum term-by-term without combining the results. [Note that 
the log x term in line 10 gave rise to the term in line 12.) To our 
rescue comes a simplification program RATSIMP [HATional Sttlplifl- 
cation) which will expand: denominators and combine the entire result 
into a single fraction. This fraction is simplified by removing the 
greatest common divisor of the numerator and denominator* 


(13) J RATS IMP (Input) 

fill 7 (response). 

x + tx + x 

Here is another integration problem; 

(15) x**3/(l - k** 2]“{3/£) (input) 

(16} (response) 

C*-*2) 3/Z 

(1?) ‘integrated^, x) (input) 

&qrt(-x ? + 1} + - - - (response), 

sqrt(-]r + 1] 


Note the system's preference for -x 2 + i over the more convention¬ 
al 1 - x 2 . Mathematic jane today tend to obey the rule; If a sum of 
two terms contalais a positive term and a negative term* write the 
positive term first. Graphic heuristics of this sort are now included 
In Martini more sophisticated display routines. 


09) 

{2*x^+5*X*M+x**3+4*Jt**2+l )/{* 

**l+ l)*"2*e**x**2 
(input) 

C?D) 

2x 6 + &x 4 + * 3 + M *2 

(res pome) 


(* 2 + l) 2 

(£ 1 } 

'fntegratet'kS* x) 

(input) 

(22) 

2k + 2x J + t c x 2 

(response). 


2k 2 + 2 


These results depend on a powerful pattern-matchfe^ program for 
algebraic expressions. 
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This example pointu clearly the need (or a better way to enter 
mathematical express ions into the system* Methods that allow users 
to hand-write algebraic expressiona, using a pen-like device* are 
now close to the step o£ useful application (Ref. 7) + 

The following example shows the system solving a linear differential 
equation with constant coefficients, a problem-type of great interest 
to electrical engineers?. 


f23) 

DERlV(y *k *3) + A,*DERJV(y*5i) - sf n{2*X> 

(Input) 

(24) 

“V* = sl«U») 

Di< J w 

(response) 

(25) 

■t&FSOLVECwS* y* x) 

(input) 

(26) 

NEED INITIAL CONDITIONS 

(response). 

The 

W), 

done 

program allows one either to enter specific initial values 
Y l (0), and Y"(tl) or to leave them ae indeterminates (aa 
below)* 

(27) 

ALIFORMAL 

(input) 

(2&) 

IS THE EXPRESSION 



A 

TO BE CONSIDERID POSITIVE* NEGATIVE OR ZERO 


(response). 

As before, the answer to this [question will be used to quorate an 
answer which contains no complex terms* 

(29) POSITIVE (Input) 

(3a) ar (o) » arto) * \ 

2A 

♦ -HW » 4f(D) * i C05[sqft(<)lll 

ff - 4A 

, r(c) ^ cost2 ,) 

(response}. 

The solution is obtained with the help of a subsystem that takes 
Laplace transforms of both sides and obtains the inverse Laplace 
transform of their ratio, using the package for integrating rational 
functions. 
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Hierarchical organisations such as MATH LAB h s will become increas- 
Lngly popular as programs and systems get more complex and so¬ 
phisticated. While this system is ostensibly working at the higher 
levels oi the hierarchy it is actually spending most of its time at 
the lower supporting levels. Probably, over hall the routines of its 
more than 60*000 words of memory are being utilised in solving a 
differential equation. This should serve as a warning to designers 
of time-sharing systems who would prefer that programs limit their 
use of memory. 

The field of computer-aided! instruction (CA1) has not progressed to 
the extent that masiy have wished. Teaching programs are unable to 
answer questions other than those that the designer had foreseen -- 
not only because these programs do not know enough English to un¬ 
derstand the [juration* but basically because they do not at all under¬ 
stand the field about which they are supposed to teach. The designer 
of the usual kind of CAI course writes a script which is controlled 
try a context-independent interpreter. The script designer must pro¬ 
vide for many possibilities at each step because he cannot rely on 
the interpreter of the script to know anything about the field with 
which the script deals* If a student is allowed* ae he rarely is* a 
reasonable flexibility In replying to the program* then either the 
script designer is physically exhausted from considering all the 
plausible replies* or he misses considering many such replies. We 
contend that a primary step in writing a sophisticated teaching pro¬ 
gram should be the education of the program in the area about which 
it must teach. 

Consider, for example, the problem of writing a program for the 
teaching of freshman calculus techniques for differentiation and inte¬ 
gration. Using the knowledge of these techniques that is embedded 
In the WATHLAB system. It would be possible to write teaching 
programs that check the steps of a student^ calculation, advising 
him of errors as he progressed. A preliminary program with these 
facilities has been prepared, Such a program could also be capable 
of performing differentiation or integration problems suggested by 
the students* explaining its steps as it proceeded. To achieve such 
capabilities through the writing of mindless scripts is unthinkable. 

To be sure, the experiments with the Calculus Rate Problem Solver 
(CARPS) have shown that a groat deal more must be known before 
such programs can deal effectively with human beings in a wide 
area of knowledge* 

The task of finding out what human users know is clearly related 
to the tasks of workers In the field of linguistics* psychology* and 
philosophy, A major falling, however, in the education given in these 
traditional disciplines is the lack of understanding of the concept of 
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an algorithm or process. Wo believe that advances in our under- 
standing of human knowledge as algo-rtlhmic processes are likely to 
be of supreme importance. 

Although the moat essential part of MATH LAB is conceptual and the 
next, most essential is the programming of symbolic transformations, 
the interface with the user is> very important, and we look forward 
to giving MATHLAB every advantage of effective man-computer- 
Interaction techniques. 

Chess 

MACHAC-VI, a chess program developed by Richard D. Greenbfatt 
for the PE>F-& computer, was begun in November 11)66, It entered 
its first tournament the following January. Since then, it has par¬ 
ticipated in lour more official 1LS, Chess Federation tournaments 1 ' 
and has achieved an official rating of 1628. In its last tournament, 
it had a performance rating of 1720 and drew an 1360 player. These 
statistics put It a little more than one standard deviation below the 
mean strength of all U.S. tournament players (which is about 1830), 
The machine has played perhaps 3000 games against chess players 
of every streiigth. it wine about 86 per cent of its games against 
tournament players, tareenblatt estimates that it represents a pro¬ 
gramming effort of about six man-months. 

Once the basic program was completed, the first step in refining 
its play was to give it a model ol the flow of power on the chess 
board. Next, it was given explicit techniques for pins and discov¬ 
eries; then it was given positional and pawn structure considera¬ 
tions. These and other features arc largely completed now. 

The next major step, only partially Implemented now, will be to in- 
corporate symbolic reasoning ability into the program. The oppo¬ 
nent's threats are specifically identified, and each reply is evaluated 
for its ability Lo answer some or all of the threats. This docs not 
moan that the opponent's threats were ignored before; but now the 
program proceeds with a more explicit sense of what it la doing, 
whereas in the classical minimax strategy the threats arc implicit 
and therefore cannot be subject to symbolic statements of what to 
do, The explicit system should be much more efficient. 

Another new feature, already implemented, adapts the plausible- 
move system to recognized stales of the game: 11 one side appears 
to be ahead In, the look-ahead analysis, it will prefer holding, trad¬ 
ing and simplifying moves; the other side will require positive, at¬ 
taching moves that might yield tangible gain. These and other fea¬ 
tures are operational. Out the other programs have not been modi¬ 
fied to take as much advantage ol them as they probably could. 
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Another class of improvements is planned which will include real 
symbolic learning rather than mere tuming-up. We expect tiiat these, 
with improvements in end-game strategy, will result in a substantial 
increase in strength of play. 

Eye-Tracking 

The much-used term 11 man-machine interaction" usually refers to 
situation^ in which the communication relationship is exceed!ngly 
lopsided, Man can see, hear and touch the computer in many ways 
acid, places, but the machine is restricted to receiving information 
through a narrow bottleneck — usually a Teletype, We have been 
interested for a long time in redressing this imbalance. This year 
we were able to achieve an old goal of enabling the machine to loot 
at a person. More precisely: the machine looks at the man's eyes - 
to determine his point of fixation, 

Recordiiig eye movements is a well-established technique in the 
study of perception, tracking skills, and even problem-solving be- 
liavior. But traditional methods give the pattern of eye movement 
only alter analysis. Wc believe our system, which incorporates an 
eye-tracking device developed by J. Merchant of Honeywell under 
NASA contract, is the first that enables a computer to use the in¬ 
formation in real time. As an example of its uses, Samuel L. Gcff- 
ner lias two programs that display text for a subject (such as a 
small child) to read and take action related to the word currently 
fixated. One of the programs causes the word to be pronounced by 
the computer; the other causes the word to be replaced, in the dis¬ 
play, by its translation into anotlier language. These programs are 
mere demonstrations --but they illustrate rich applications to 
teaching and other interactive situations. We are pursuing such ap¬ 
plications, partly with support from NASA. 

The AJ. Ti m e-Sharing System 

The experimental work of the Artificial Intelligence Group requires 
a high level of computer service for a limited number of users. 

Our system is based on time-sharing a two-processor machine 
(PDF-6 and PDP-10) with a core memory of 2 Le words of 36 bits. 
Normally, the programs of the active users remain in last memory; 
this limits the number of users now, but a paging device to be com¬ 
pleted early in 1970 should allow some expansion required by the 
maturation of certain projects, notably MATHLAB. 

The special requirements of the project led to a time-sharing sys¬ 
tem with enough novel features to merit discussion. Donald E. Ehsi- 
iake describes the system in detail ("ITS 1.5 Reference Manual % 

A.L Memo 16la)* Besides the usual kinds of input-output and system 
calls, there are special calls to reduce overhead and to facilitate 
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real-time control. These Include programs for operating the muchan- 
ical hands and computer eyes and other special remote-control de¬ 
vices, All ordinary time-shared programs run in one of the proces¬ 
sors f and critical real-time processes are assigned, by user calls 
to the other, The real-time processor is normally assigned to a 
single user at a time* 

Because all user programs ordinarily reside in core, it is possible 
to switch between programs with great rapidity. Quanta of user 
time are short enough to allow program response to single typed 
characters without noticeable delays. When swapping is introduced, 
we expect it to affect only semi-dormant users, 

A user may have many job$ running "’simultaneously". Each user 
commands a tree of procedures; each can create and control sub¬ 
ordinate procedures; all have equal access to external devices and 
files. The top procedure, loaded automatically when the user de¬ 
clares hw existence, contains a version of the well-known DDT de¬ 
bugging system, A special character allows transfer to the top pro¬ 
cedure from any level,, so that the user is automatically fn position 
to use DDT’s interrogation, breakpoint, and other debugging features. 

Input-output devices arc referenced symbolically and data can be 
transferred on character, word or block bases; an entire video 
ima^e can be acquired by a system call while the calling procedure 
continues without waiting. User programs need no butters In their 
own core images* Line-printer requests, for example, are buffered 
until the device is free* User programs communicate by "software 
interrupts" that are treated the same as hardware interrupts; superi¬ 
or procedures can store and retrieve words in inferior procedures 
as though they were t/O devices; buffered communication is pro¬ 
vided so that pairs of procedures can treat each other as input de¬ 
vices. The file system resembles the CT3S file system* 

The schedule algorithm tries first to equalize time between users, 
and second to equalize times between the procedures of a single 
user tree. A special procedure always runs which performs various 
jobs and can check constant portions of the system against a copy 
in ait attempt to detect some forms of hardware or system failure* 

The secondary storage uses IBM 3311 discs and DEC microtape 
drives which are file-structured in the same way, Users can estab¬ 
lish symbolic links between file directories so that files can be 
shared by many programs and many users; these links can be 
chained* 

The system lias several operating stations. These include four text- 
display devices, several Teletypes and external telephone lines, a 
main console with DEC 340 display and several slave monitors 



around the laboratory „ and a special console with controls for oper¬ 
ating the eye-hand system. The system interfaces with a radio trans¬ 
mitter for Inexpensive long-distance remote experiments, 

A multiplexed digital-analog system operates either in word or block 
mode, on call or automatically, as requested. Any number of users 
may simultaneously have analog access on different channels in dif¬ 
ferent modes, Those,are some of the real-time faculties: 

1) Iris, focus, stereo mirror, lor high-resolution image dis- 

seetor, 

2) Pan, till, zoom, focus, iris, lor email, low-resolution 

image dissector, 

3) Extend, tilt, rotate, grasp, finger curl, for hand MA-3, 

4) Extend, tilt, rotate, grasp, for hand MA-2, 

5) - Roll, yaw, horizontal, vertical^ swing, lor arm MA-3, 

6} Position for tactile-sensor device, 

7) X, V, Z, rotate for arm MA-3, 

For moving the arms, special system calls provide acceleration- 
arid velocity-limiting, software limit stops and other performance 
limits. To prevent disaster, certain motion commands are Lminter- 
ruptable for limited times, and an arm-moving call to illegal if the 
device is under control of another user. Motion calls are not auto¬ 
matically buffered like those to the line printer] Although one can 
road any input channel at any time, most mechanical devices can 
be operated at leisurely speeds, with the Input multiplexer channels 
read automatic ally. The system normally does this at a minimum 
Of 3 times per second. 

On the input side arc sensors for all mechanical degrees of free¬ 
dom, including hands and arms and optical parameters* There is a 
teat stand for calibrating the positional control of the arms, and 
there is a variety of portable remote-control boxes with switches 
and continuous-adjustment knobs, A special system call gives the 
user great flexibility in real-time control of program parameters. 

It connects potentiometers {through the input multiplexer) to arbi¬ 
trary program variables — Le*, memory registers -- and these 
can be specified to be fixed or floating, or arbitrary bytes. There 
are options for assigning absolute or Incremental meanings to knob 
positions; in the incremental mode, there is a programmed velocity- 
dependent gain and a side-to-side hysteresis so that it is easy to 
make fine and. coarse adjustments with the same control, and it 
tends to remain centered in its motion range* 


as 


3* 


TJic System contains pood facilities for graphic display, which are 
presently limited to one user*. Growing needs dictate acquisition or 
multiple-display hardware. 

LISP (MACLISP) 

LISP is the high-level language the Artificial Intelligence Group 
uses. Our LISP version is different in many ways from other 
LIS Pa p and some of these differences represent progress. The de¬ 
tails are of interest only to specialists, who may consult the memo 
by Jon L. White {"Time-Sharing LISP for the PDP-G*, AX Memo 
I90) + The system does not use am A-list for ordinary variable bind¬ 
ings; It uses a value property; a special stack is used to unbind 
after lambda-conversions. There are many small improvements in 
human engineering such an default values of non-specified function 
arguments. The garbage collector manages apace for arrays. The 
READ program allows new character-handling facilities and macro 
definitions of certain kinds, There is a variety of powerful editbig 
facilities and display packages, and some strong debugging systems. 
The machine-language feature, LAP, has been strengthened. Whit¬ 
field Duffle has restructured the compiler to make the most of Its 
decisions by dispatching on tables so the users can introduce special 
forms by giving the compiler instructions through the tabic entries. 

It would be more efficient tf the compiler were able to make a 
deeper and more systematic analysis of the uses of mtpresnions be¬ 
ing compiled, separating executions for value from those for effect 
on flow of control, for side effects, etc,, and for recognition of un¬ 
necessary recursion and similar simplifications. This itself is a 
problem in artificial intelligence. 

.Motivated by MATHLAB and other projects, we are now making an 
effort to introduce facilities for very fast arithmetic into LISP, We 
expect that the resulting system will be almost as efficient at num¬ 
ber-crunching computation ns is any conventional algorithmic lan¬ 
guage, 

Mechanical Hands and Arms 

The project has developed several mechanical effector devices for 
computer-controlled manipulation. This field is still substantially 
unexplored, and our work can be considered only to clarify some 
of the problems, not to solve many. There are problems fn several 
different areas. 

There Is a need for much deeper analysis of what is needed in de¬ 
signing a hand-arm system for various kinds of jobs. In his thesis 
David L. Waltz studies motions the human hand uses lo perform 
several basic tool-handling and similar tasks, to see wlilch of the 
degrees- of freedom of the band were mo3t essential, and how they 
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were sequenced, There ig gome information about this in the ortho¬ 
pedic literature, but it is not sufficiently structured to serve as a 
base for programs. One needs to describe actions Iti terms gf inter¬ 
actions of position, force, velocity and setts ory responses “-in short, 
one needs something like a programming language for ft. We also 
examined some choreographic languages but, although they contain 
some good ideas about path-of-motion description, they do not have 
sensory-interaction predicates. Walt?; developed some notation that 
seems helpful and Ernst's (Ref. 0) discussion ts still relevant, 

Waltz s (earlier) thus is shows how a few degrees of freedom can 
accomplish much, but one wishes there were much more known about 
this subject. 

In most of our experimental work, we have employed a modified 
AMF Versairan industrial manipulator arm (which operates in cylin¬ 
drical-polar coordinates). Although the hands w& attached to this 
device have some extra motions, the large-scale motion of that type 
of arm has no such redundancy; thus there is basically only one 
way to reach each point in space. Calculating this is straightforward, 
solving a not-tao-complicated equation that has only one solution. 

Because this is a very clumsy device — such an arm is unable to 
reach around obstacles or even to support an unobstructed object 
from an arbitrary direction — we developed a much more mobile 
arm with many extra degrees of freedom. The new arm behaves 
more like a tentacle than like a coordinate system. Hydraulically 
operated, it is relatively slim and has five articulation, each with 
two degrees of freedom, a shoulder, three elbows and a wrist. An 
articulated prototype shows clearly that this geometry is adequate 
for everythhig a human arm can do and more. In the actual hard¬ 
ware mechanism, each of the eight Interior hinge joints has approxi¬ 
mately 110 degrees of flexion. We conclude that this is inadequate; 
it must be more nearly 180* The lesson we learned is that in a 
minimal system a small angular restriction means only that the 
work space ts somewhat reduced. But, in a linkage whose purpose 
is a great variety of ways to reach points in the work space, all 
restrictions interact in messy ways to break up the higher-dimen¬ 
sional mobility space into hard-to-understand fragments, ho that two 
slightly different throe-dimensional positions may have to be reached 
(if at all) by grossly different global arm configurations. For such 
a mechanism with 10 degrees of freedom for reaching points In only 
three dimensions, tfiere is a mathematical problem of inverting the 
position equations, One can sometimes get. by with simple hill- 
climbing by successive approximations, but this i& really unsatis¬ 
factory because it is hard to adjoin global information, and it docs 
not give an analytic picture of the alternative solutions to the prob¬ 
lem, Jean-Yves dresser, in bis (earlier) thesis discusses a method 
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that gives a solution specific to our arm - mechanism, using a ratiior 
general method of dividing the position problem into a scries of 
almost-independent factor sub-problems. Wliat Gresser does is to 
divide the arm in half, and describe the mobility space of each hall, 
using simplifying approximations for each side. When the half- 
solutiqns are pul together, there needs to lie an accuracy-increasing' 
iteration anyway, so there is hardly any real loss through approxi¬ 
mating. 

Now, in our arm-mechanism, it happens that the mobility zones of 
the half-arms can be described In terms understandable to humans -- 
for this particular arm, each zone resembles a portion of a torus * 

To find the ways to get the whole arm to a certain space position, 
one describes the intersection of the two tori, one centered at the 
slioukler position and the other the goal position. Each intersection 
point yields an approximate solution to the problem (that is, a pos¬ 
sible location of the center of the arm), and a higher-level program 
couid bp. asked which solution is most desirable. 

There is 4 larger problem here; how should otic represent 4 ma¬ 
chine’s body image? For the problem of a single, not-too-complicated 
4 rm, one can doubtless get by with cleverly coded, sparse* three- 
dimensional arrays, but one would like something more symbolic, 

And one wonders what happens in the nervous system; we have not 
seen anything that might be considered to be a serious theory. Con¬ 
sider that a normal human can place an object on 4 table, turn 
about and make a gross change in his position and posture, and then 
reach out and grasp within one or two inches of the object, all with 
his eyes closed! It seems unlikely that his cerebellum could per¬ 
form the appropriate vector calculations to do this; it is not a mere 
matrix inversion (and, even if it were, one would still need a theory)* 
We would presume that this complex motor activity Is made up, 
somehow, of a large library of stereotypical programs, with some 
heuristic interpolation scheme that fits the required action to some 
collection of reasonably similar stored actions* But we have found 
nowhere any serious proposal about neurological mechanisms for 
this, and one can only hope tint some plausible ideas will come out 
of robotics, research itself. Unfortunately, at present this area is 
somewhat dormant. 

Another conclusion from our experiment? is that delicate manipula¬ 
tions must be controlled by application of controlled forces (rather 
than by direct position control), with attention to matching velocities 
with inertias. For measuring the forces on a mechanical hand, we 
have taken two approaches. First, it is quite feasible to engineer a 
grasping surface with good pressure sensitivity at a great many 
points by wrapping a coaxial cable connected to a Time-Domain Re- 
flectometer* (A TDK is a sort of radar system designed to measure 
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reflected radio waves that arc produced In a soft-sheathed tube* by 
any deformations of the wall. The instrument permits hundreds of 
thousand* of points, using a single electrical connection to the hand. 
(Waltz describes this system in his thesis.) Second, we have devel¬ 
oped a strain-gauge telemetered wrist that provides force informa¬ 
tion in the necessary six degrees of freedom. This little instrument 
is able to tell where and in what direction the hand is being pushed, 
assuming that the force is being applied at a single point — a con¬ 
dition that usually holds in a first contact with an object. 

Solution of the appropriate moment equations for the set of six 
forces at the wrist (Fig. 3-C?) yields a certain line in three-dimen¬ 
sional space, along which a force of a certain magnitude ie applied. 
If the machine knows the geometry of Its hand* it can presume that 
the force is applied where this line intersects the hand. 



Fig* SO 


We have built this hand and are testing it. We shall describe it in 
a memo by Callalum, Shah and Minsky, 

Computer EJyes 

A computer must have eyes if it is to see. When the project start¬ 
ed, no device was available for interfacing a computer with a real¬ 
time camera, although there were film-reading devices on the mar¬ 
ket. We first used a Vidtcon television camera* but we had diffi¬ 
culties with the limited signal-to-noise quality of the Vidicon. 
Besides, Vidicons are not suitably adaptable to random-access 
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scanning, they have motion persistence and other problems. We 
decided that this added up to too many obstacles. The image- 
dissector camera offered much cleaner images {at the expense of 
sensitivity, which, for laboratory purposes, was unimportant) so* in 
collaboration with Information International, Inc*, we designed and 
constructed our present camera system. An image dissector is es¬ 
sentially an inverted cathode-ray tube: an image 'projected on a 
photo-cathode produces electrons which are focused through a de¬ 
flection system so thal the current from a selected image point 
falls through a small holt! and is then measured. 

In such a system, the signal-to-noise ratio depends essentially on 
the number of electrons t and one can obtain more precision at the 
price of longer time-exposures. We decided to put this signal-to- 
noise ratio under direct program control and to make it independent 
of the brightness of the point being measured. This is built into 
the camera's video processor, Horn gives the details {'The Image 
Dissector 'Eyes 1 % A.I. Memo 176). If the constant signal-lo-noise 
constraint were enforced even on very dim points, the exposure 
times would become excessive, so the system has also a dark cut¬ 
off parameter that terminates the measurement very early if an 
initial estimate of the brightness falls below a specified value. This 
is also built in the video hardware. The resulting system can meas¬ 
ure intensity over a 4096:1 dynamic range, with a brightness- 
discrimination sensitivity of one part in 64, throughout this range. 

The output, appears as a floating point number or as a logarithm 
of intensity. The eye has also a reference-brightness input that 
can be used to make the measurements almost Independent of over¬ 
all illumination fluctuations, and there arc various protective de¬ 
vices to prevent damage to the image tube by excessively bright 
light sources, Horn's report discusses many advantages and pitfalls 
in using this system; anyone contemplating using image -dissectors 
for computer vision should correspond with us about the results of 
some modifications now under construction. 

It is frequently suggested that one ought to build various forms of 
parallel-processing artificial retinas, perhaps along the lines of 
biological systems. But the operations of the vertebrate retina and 
the subsequent image processing is not nearly so well understood 
as ia generally believed, and we do not think the time is quits ripe 
for taking such a step in liardw&re. In our present operations, the 
computation time consumed in the higher levels of scene-analysis 
is comparable to that spent in the lower-level pre-processing, so at 
present there would be no large time factor to be gained from fast 
parallel hardware -- even if we were abie to decide how it should 
work,. 
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Res parch o n intelli g ent A uto mata - Marvin Minsky and Seymour Paper* 

T3io largest sector of research In our group is still the study of methods 
for providing machines with greater visual and manipulative capabilities. 
Our general approach to this goal was described in last year's MAC 
Progress Report* We expect, some time in 1969, to demonstrate some 
practical capabilities of automatic, visually guided manipulation, by shotting 
the computer a structure made of children's blocks, and having it build a 
functional duplicate. Details of this work are described separately in the 
Status Reports of the Intelligent Automata Project, and in many of the 
Artificial Intelligence group memoranda available through our office. 

Com p utational Geometr y - Marvin Minsky and Seymour Paper! 

Preoccupation with theoretical aspects of machine vision has led us to 
crystallise a general concept we call ■computational geometry". This is a 
now mathematical specialty concerned with the complexity of computations 
necessary to recognize various properties of geometric objects. The rapid 
evolution of discoveries in this area has given us hope that it will lead to 
new directions for "computer science* in general, by providing subject 
matter that combines intuitive clarity and practical importance, with close 
connections between classical parts of mathematics - geometry and algebra* 
We believe the widely recognized conceptual fragility of current ’'theories* 
about computation is due in large part to their attempt to build upon ex¬ 
cessively abstract principles of automata theory and linguistic structure, 
without enough concern for thorough understanding of particular problem 
areas in relation to particular machine structures.. The following sections 
show some examples. 

A. The SEE Program 

Computational requirements lead to geometric questions of mi entirely new 
sort, such as surely never occurred to Euclid. Consider the need to analyze 
a scene in which some of the objects partially obscure others. 



The methods riescribed in last year’s report presupposed a computer model 
or description of each kind of object - cube, pyramid, etc. Since than it 
has become clear, in further work of Adolfo Gusman, that this information 
is not wholly necessary. Indeed, computations are usually much simpler if 
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based on a more abstract and general theory of the appearances of objects 
Earlier methods were based on partial recognition of the bodies such as 
the scene below; 



Here, where all objects are rectangular solids, and do not occlude one 
another badly, we can discover the objects by the extremely local process 
of locating all the *Y~joints,* Each object contains at most one such dis¬ 
tinctive feature. "lhLs could, of course, fail because of perspective, as in 

□ 

which could be a cube, or in 

0 

ftor we require each of the three angles oi a ¥-joint to span less than 
100 degrees^, A more serious failure is in the case oi occlusion, as in 


wlierc one of the Y-joints is completely hidden from view. But the great 
power of programs capable of hierarchical decisions is Illustrated by the 
possibility of first recognising the small cube , then removin g it. then 
e xtendin g the hidden line s, and so discovering the large cube! 



The program developed by Adolfo Guzman proceeds in a rather different 
way, his idea is to treat difiorent hinds of local configurations as providing- 
different degrees of evidence for fl linking’ the faces that meet there. For 
example, in the following three types of vertex configurations 


U 



















the "Y" provides evidence for linking I to H, It to EH, and I to nr. The 
* Arrow" just links I to 1L Because a ’"T" is ordinarily the result of one 
object occluding part of another, it is not regarded as evidence linking I to 
IH, or IT to in {and it is also neutral about r and II), Using just these 
rules, we can convert pictures Into associated groups o£ (aces as follows: 
we represent Y-links by straight lines and arrow links by curves. 





{4) 


So far, there has been no difficulty in associating sets of linked lacea with 
objects T The usefulness of the variety of kinds of evidence shows up only 
In more complicated cases, in the Example f5) 
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f 5 6 of vertices m different ob- 

jMts + To break such lalse connections, the program usee a hierarch teal 
scheme tliM first finds subsets of face* that are very tightly linked (e g 
by t*a w mere lb**). These "nuclei- then eompete for more we^l! 
linked faces There is no competition in Examples but in fSJ the 

ample (6), if a very aimple "competition" algorithm were not adequate here 
ene could also take into account the negative evidence the two Taints * ' 

provide against linking I-m and H-m 




ZLTllTT^t 0nlV ^ 36:6144011 of hls seh ^ Guzman uses several 

ether kinds o( links, Including evidence from eollinear T-joiut linos of the 



and the effects of some vertices are modified by their associations with 
others Th ig variety of resources enabLes the program to solve cornel, 
scenes like that illustrated at the beginning of this section, 

™i Wil \ be in Gu * minr * doctoral thesis to appear in the Spring of 

i®*?' Sur P riS[n ff and elegance of this algorithm suggests that 

there will evolve a rich theory of the geometry of object clusters. Further 
evidence or this comes from the discovery of simple h.uristi^ t J ^em 

to generate plausible hypotheses about missing lines in pictures of com¬ 
plex scenes. 


B. The Theory of Fereoptrons 

fnv We haV * b(?6n ™ terEf&ted in finding a theoretical basis 

ab6m Sl and llmitatidn s ^ the peresptron and similar ma¬ 
chines. These are highly parallel computation schemes that attempt to 
recognize complicated inputs by l) computing many properties of the input 

reHthTri/ql' T^ ^ rec ^ nix ^ and then 3) basing a decision on SOme 

relatively simple combination of the results of the first stage such as a 

23SE35I 1 ° f Weifih,ed &UniS ° f eVidEnCC f0r eaeJl ““■»**« alternative 
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Our interest tn such machines jg not based on very much concern for their 
practical possibilities, for these are very limited. However* It in our con¬ 
viction that these machines are nonetheless of critical importance ns theo¬ 
retical models T because if we cannot thoroughly understand such simple 
computers s we tan have little hope of obtaining good theories of more. 
powerful and general computers, (Thu "theory of computability- for ■"uni¬ 
versal 1 ’ machines is totally unsatisfactory in the context of any real prac¬ 
tical problems.) Fortunately* we have obtained a wide variety of theoretical 
conclusions about perceptions* and these are given tn detail in a new book 
(Perceptio ns; An Intr od uction to Com putational Geometry . Marvin Minsky 
and Seymour Papsrt* M.r.T. Press,, 1069), 

We shall summarize some of the results here. First, let us define a 
perception of orde r jfr - 

Let R be a part of a two-dimensional, plane. Let X be an arbitrary black- 
and-white "picture", i,e t) a subset of R. Any point in X is considered to 
ba black, and the rest of R white. Let $ be a set \$ t , . .j 0 f 

predicates - functions whose values are 0 or 1 - each of which depends 
on no more than K points. Finally, choose for each £ a number a . and 
define ^ 

tm = 1 U Z < 0 
* 0 if < 0 

This definition includes the concept of "threshold"* as in 

I^CD — ^ (£ iny real number) 

if we allow one of the $ functions to be a constant. 

Now we ask whether a perceptron can recognize a "pattern". For example 
is there a perception such that *(X) - 1 when X Ls a square (or convex, 
or connected) and ^(X) = 0 when X is not a square (or not convex, or dis¬ 
connected) 7 Our analytic methods depend mainly on replacing the geometric 
concept of a pattern by the algebraic properties of the transformations that 
preserve the features that concern us. We cannot give a full example of 
how this is done for geometric concepts, but the following sketch shows 
how the algebraic theory works in a fairly simple case. Chapter references 

point to corresponding sections in the book Ferea ptrong. 

Theorem 3,1 (Chapter 3) I nformal Ve rsion 

Suppose the retina R has a finite number of points. Then there is no per- 
ceplron > 8 that can decide whether or not "the number of points 

in X is odd- unless at least one of the $'s depends oil all the points of R, 

Thus no bound can be placed on the orders of perceptions that compute 

this predicate for arbitrarily large retinae. To realize it, a perceptron 
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must start with at least one $ that looks at the whole picture] The proof 
uses several steps, 

Step 1; In n.l-Sl,4, we define "percepiroo", "order", etc*, more precise- 
ly, and show that certain details of the definitions can be changed without 
serious effects. 


Step 2: In § 1.3 wc define the particularly simple f functions called "masks' 
For each subset A of the retina, define the mask to have value 1 if + 

ihe figure X contains or "covers* all of A, value Otherwise, Then we 
prove the simple but important theorem (fl.5) that if a predicate lias 
order > K (see SI.3) in any set of $ functions, then there is an equivalent 
perceptron that uses only masks of si&e > K (see 3 0*2)* 

Step 3: To get at the parity - the 'odd-even" property - we ask: What re¬ 
arrangements of the input s pace R leaves It una ff ected? That is, we ask 
about the group of transformations of the figure that have no effect on the 
property. This might seem to be an exotic way to approach the problem, 
but since it seems necessary for the more difficult problems we attach 
later, it is good first to get used to it in this simple situation. In this 
eutu., the group is the whole permutation group on H - the sot of all re¬ 
arrangements of Its points, ~ 


Step 4; In §2 wo show how to use this group to reduce the perceptron to a 
simple form, Thcjgrgup ^invariance theore m proved in 32,2 is used to show 
that, for the parity perceptron, all masks with the same support size * that 
fSj-^IMrhosc that lo ok at t he s ame num b er of po ints - can'be given identical 

coefficien ts* Let 0, be the weight assigned to all masks that liave sunnort 
size ■ j. 1 



> °1 


Group-invariant coefficients for in I = 3 parity predicate. 





































Step 5; It is then shewn (in §3,1) that the parity perceptrun can be written 
in tlie form 

hi('f) >», 

0 

where jx | is the number of points in X, k i£ the largest support size, and 
|^:X|^ the number Of subsets Oi X that have j elements,. 

Step 6: Because 

( I J * ^ ( n)(n - l) + .. + (ja-j* l) 

is a product of j linear terms, it is a polynomial of degree j in n. There¬ 
fore wu can write our predicate in the form 

(1x|) > 0 3 

where P-^ is a polynomial in |X| of algebraic degree £ k, Now If fXI is an 
odd number, P|-(jx[) >0, while if 1X| is even, P^flXl) c 0, Therefore, in 
the range 0 S !Xt £ |Rl, must change its direction |R| - 1 times. But 
a polynomial must have degree > |R| to dp that 7 so we conclude that 
k > lilL This completes the proof. 



This shows how the algebra works into our theory* For some of the more 
difficult theorems we need somewhat more algebra and group theory. 

Here are some of the positive results; that certain patter ns have curtain 
orders. 

P atterns oE_ Or tier _l 

§0+8 * The center of gravity lies to the left of a curtain given point on the 

X-axis." 

§2.4 Other similarly defined properties of moments, in fixed coordinate 
sy stems . Includes "The area of the image is less than A,." 


§1.5 


Linear threshold inequalities. 















■1A "The image is exactl y a certain one", qt "differs from it by not 
more than a given area A." 

Patterns of Order 2 

57.3 -The figure is symmetrical shout a fixed point in the plane,” 

3Ti9 " TlA '° fl S ures ' on ^ Bivan Maes, are congruent under translation/ 
(The coefficients diverge, however, as the retina size grows.) 

51.6 "The area of the image lies in a certain range." 

■b,2 "The figure lies within an axis-parallel line." 

50,8 "The moment of inertia of the figure exceeds some threshold." 

Patterns of Order 3 

17+10 "Two figures on two given lines art? translation-equivalent with 
hounded coefficlouts Wj/ 

§0.3 "The image is a convex figure." 

ffl T $ "The image is an axis-parallel rectangle?/ 

§T ” 7 'The image is a square/ (Coefficients diverge) 

56.3 Any two instances of figures not translation-equivalent can bo dis¬ 
tinguished, 

Patterns of Ord er 4 

UJ " T]lt i£ * square parallel to the axis." (Bounded coefficients.) 

§6.4 "The linage is a circle." 

§7.5 "The image has a vertical axis of symmetry." (Coefficients diverge) 
(Believed to be unrecognizable in any order for bounded coefficients) 

§7.4 Reflection symmetry on a line. 

55,8 "The Euler Number - that is, the number of Components minus the 
number of Holes - exceeds a given number. 

Patter ns of O rde r 5 (or possibly one less) 

58-3 "There is a certain number of convex objects.” (No holes permitted) 

5 0,3 "The total curvature of all the buundas-ies lies in a certain interval/ 

§ 7.5 -Two plane figures are equivalent under translation/ (Unbounded 
coefficients) 

Patt erns of Order 6 (probabl y) 

S7,3 "Two plane figures are equivalent under translation and dilation/ 

The remarkable thing about the above results Is that the order remains 
fixed, regardless or the size of the retina. But for other patterns, the order 


















Increases without hound as the retina is made larger, and it is lair to 
say that these are, in a practical sense, outside the conceptual ability of 
pe r c eptrgn-like machines- These includes 

Pan creas oi Unbounded Order 

i3.1 "The number of occupied retinal cells is odd. 1 ' (The order is known 
to he equal to the number of points in the retina.) 

5 5 "The image is a single connected whole*" (The order ia known to 
grow faster than a/n, probably grows as yN.) 

5 5.£f All topological properties except, simple functions of the Euler Number. 
For example; "one pari of the image lies within a hole inside another 
part* cannot be recognized. 

|4.0 Certain simple Boolean combinations of pairs of first-order patterns. 
This and §3.1 show strikingly how perceptrons differ from serial 
computers* for these patterns are very easy lor serial machines. 

§0.6 Very few of the finite-order properties mentioned above remain finite 
order in the context of other figures, or of noise. 

C r Connectedness and Serial Computation 

Our deepest results in the perceptron theory are concerned with the geo¬ 
metric - or* rather, topological - properties of connectedness . Because we 
felt that there was some Inherently serial - or recursive - character to 
connectedness, we decided to investigate the computational geometry of this 
concept in the context of other mixtures of serial and parallel machine 
structures, Including Turing Machines and Iterative Arrays. Some of these 
are discussed in Chapter 0 of Perce ptrons. We can give here an example 
of one such result, 

Terry Beyer has investigated the time necessary to compute ^ e en nE c(sa in a. 
situation that provides a different and perliaps more natural model for 
parallel geometric procedures. Suppose that each square of a retina con¬ 
tains an automaton able to communicate only with its four neighbors. It can 
also tell the state (black or white) of its square, The final decision about 
whether the figure is connected or not Is to be made by some fixed automa¬ 
ton, sav the one in the top left-hand corner. On the assumption that the 
states change only at fixed intervals of time, we ask how many time units 
must pass before the decision can be made. It is obvious that on an nx n 
retina this will take at least 3n time- units, for this is the time required 
for any information to pass from the bottom right corner to the top left* 

It is not difficult to design arrays of automata that will make the decision 
in the order of N time units* where N is the area of the retina. Beyer’s 
remarkable result is that {2 + *)-VN is sufficient, where f can be made as 
small as one likes by allowing the automaton to 3iave sufficiently many 
states. 
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Thus the order of magnitude of time taken by the array is proportional to 
VINU which is intermediate between the times taken by the single serial 
machine (N) and the most general type of parallel computer wlitch is known 
to take < (log N) * { Perceptruns . Chapter 9) 

The following gives an intuitive picture of Bayer's algorithmic process. The 
over-all efteci is that of enclosing a component in a triangle as shown be¬ 
low, and slowly sweeping it into tlie northwest corner by moving the hypote¬ 
nuse inward. 




Each component is compressed to one isolated point before vanishing. When¬ 
ever this event takes place, it can be recognized locally and the information 
transmitted through the network to the corner. Thus the connectedness de¬ 
cision is made positively or negatively, depending on whether such an evejit 
happens once or more than once. More precisely, the compression process 
starts by finding the set of all "southeast corners' of the figure 

In this theory the retinal points are taken to be a square 
checkerboard-likc array. Two black squares are eom- 
nected if they share a common edge, or are in a ehain 
oF iicieares so connected, {Diagonal corner-contact does 
not count as a connection,) 

The centor square is a southeast corner if the Smith and 
East are empty. All other squares shown may he empty 
dr full. 



ru each step of the compression operation, every southeast corner is re¬ 
moved, and a new square is inserted when necessary to preserve connected¬ 
ness, as shown belowr 


jr 


nri 


froiild bruit 
I hr nncciHrfiJ-. 
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The diagonal lint's show how repetition of this local process dops squeeze 
the figure to the northwest. 

Repeated applications of this operation eventually reduce each component to 
a single point. The next figure shows how it (narrowly but effectively) 
avoids merging two components. 



It is easy to see that a component within a hole will vanish (and be counted) 
just in time to allow the surrounding component to collapse down. We do 
not know any equivalent process ior three dimensions. (Consider knots E) 

We believe that further investigations of this sort should lead to a deeper 
understanding of parallel computation in general and oi the potential role of 
such processes in practical artificial and biological visual systems* 

D. Designing a Stereo Vision System 

Another example of computational geometry is the reconstruction of a three- 
dimensional scene from stereo- views* In very simple cases - for example, 
where the scene consists oi a 3Ltiglc point or line segment - the problem 
belongs to perspective geometry. But scenes such as those shown In dis¬ 
cussing SEE raise different problems. One possible approach Is to make 
an analysis using REE on eacli stereo view and then match identified ob¬ 
jects. Another "pure* approach 1s to use the stereo match as a means of 
object identification, in practice, a real vision system would try to ns® a 
mixture Of both. Ell this ease a new kind of diiiiculty arises; that of mani¬ 
pulating data of varied kinds and varied degrees of reliability* 

David Perkins has been developing for his Ph + D* thesis a programming sys¬ 
tem fqr handling complex net-like data structures that could allow informa¬ 
tion of diverse sorts to be built up in the course of exploring a scene. The 
stereo problem is being used as the subject-matter for the system* 

E. Theorem-Proving Programs 

We are concerned about adapting automatic heuristic deductive programs to 
problems with large and diverse data bases. Gerald Sussman has programmed 
a classical theorem-prover based on the resolution principle and has begun 
to gain experience with the Introduction of heuristics of a more specific sort 
than are usually employed* Carl Hewitt* at the other extreme, has developed 
a system and, language called PLANNER to permit Lhe most diverse possible 
operations that might be used in proofs. Neither project is yet sufficiently 
advanced to warrant detailed analysis of results. 
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Chess and Game-Trees - Richard Greenblatl 


During the year, substantial progress; was made on Mechanical Chess - a 
domain that has long boon of great interest to workers in Artificial Intelli¬ 
gence. The program, known as MACHAC VI, has achieved the level of ability 
of an average amateur player and has been accepted as an honorary mem¬ 
ber of the American Chess Federation. The clear superiority of this pro¬ 
gram over ail previous ones is interesting both theoretically' and as a 
visible demojistration of programming tecMiques and, systems {Sec A I 
Memo 174}, 


For a long time wc have hoped to see a clarification of the theory of non¬ 
terminal Evaluation functions in game -playing, tree-search programs. One's 
first inclination is to try to interpret the E-function as an approximation 
to an estimate of probability of success, improved by the look-ahead pro¬ 
cedure. No one has been particularly successfui at making this interpreta¬ 
tion workable. Some promising' results on probabilistic strategy theory are 
given in a recent thesis by David Johnson. 


Mathem at ical Laborat ory - William A. Martin and Joel Moses 

It has been our long-term goal to develop a man-machine system for ampli¬ 
fying the effectiveness of a mathematician working on either applied or ab¬ 
stract theoretical problems. This project i& proceeding in several directions, 
to bring together such systems as the previously reported Integ ration, system' 
of J r Moses, the S ymbolic Ma thematical L abo rator y of W ( Martin, and the 
MATHLAB of C. Engelman. A two or threo-year project is planned, leading 
to an on-line system to aid in carrying out complex and tedious symbolic 
calculations, and to build up an active store of knowledge about mathematical 
algorithms and heuristic methods. We are preparing a monograph on sym¬ 
bolic algebraic manipulations. Martin is pursuing the theory of parsing 
context-free expressions, both as pare theory and in connection with an on¬ 
line parser for hand-written mathematical formulae. Moses is developing a 
simplification system of advanced power that will, for example, be able to 
convert 


to 


1 + 2 log {&ln a -B y + x + cos s By) + log'll + x) 
log (1 + x J + 1 


1 +- log (I + x) , 

With this, it should be feasible to Implement a powerful decision procedure 
far integration due to Risch; the system should be able to integrate 


log z - 2 
flog f z + t} s + 


2 . 1 

— logz + - + 


1 


log 2 s + z 
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to 


—+ logfiog a z + z) . 
log: 2 z + z 

A program, called SARGE, which drills students in freshman calculus. Inte¬ 
gration problems lias also been written by Moses. Students are supposed to 
type step-by-step solutions to an integration problem. The computer checks 
the correctness of the justification (e.g„ substitution oE variables) given for 
each step. Sometimes, though rarely, the computer can give a reasonable 
description of the type of mistake the student made in an erroneous step. 

In teractive Computer- Me diated Anima tion - Ronald M, Baecker 

Standard techniques of man-machine graphical communication have been sup¬ 
plemented with the capability for immediate viewing of a synthesized anima¬ 
tion sequence. These were constructed using three special-purpose animation 
systems which have been implemented with the Interactive graphics capability 
of the Lincoln, Laboratory XX-2 computer. 

The concept of a table-driven algorithmic synthesis of an animated display 
has been developed as an approach to the specification of picture dynamics. 
The li bit-, called movemen t descriptions , abstract aspects of behavior that 
recur over extended intervals of time in a particular animated display. 
Rh ythm descri ptions express patterns of the triggering, pacing, coordination, 
and synchronizing of picture change. 

Techniques of control that particularly exploit the interactive environment 
have been developed. The animator is coupled to the film under construction 
both through the sketching and graphical editing of static pictures of dynamic 
behavior {expressed by movement descriptions}, and through the dynamic 
mimicking of desired behavior. The animator’s dynamics are expressed 
through stylus, push-buttons, and other devices; the language may be used 
to contract animated visual displays, to build system tools that aid the 
construction process, and to implement special-purpose animation systems. 

Computer time and support for this research have also been provided by 
the M,I,T. Lincoln Laboratory, under a contract from the Advanced Research 
Projects Agency. 

Fourier Trans fo rm Methods in Imag e Processin g „ Bertiiold K, P. Horn 

The two-dimensional Fourier transform has been used widely lei optics for 
the evaluation of image-forming systems, yet has been used infrequently in 
pattern recognition and perception studies, rt was thus of interest to find 
areas of image process big to which the Fourier transform could profitably 
be applied. Although the Fourier transform has many useful properties that 
indicate its use in image processing (such as translational invariance) many 
Of these arc shared by other functions. 















The Fourier transform -was found to be useful either in spectral analysis 
of small image areas, or as a global image-transformer Lm 

l) focusing an optical device, 

.2) restoring a degraded picture, 

3J resolution beyond the usual limits, 

4J edge detection in a preprocessing pass, 

5) confirmation of edges In a given position, 

6) least-squares filtering in the presence of noise, 

7) dynamic range reduction. 

No conclusions have been reached about the usefulness of Fourier methods 
In texture description or curvature detection, A computer program was 
dev eloped for the PDF-6 to allow experimentation on images read in through 
the on-line computer "eye 11 , and the Fa3t Fourier Transform method was 
used In this program. 

Structure of Aton al Music - Allen Forte 

Some aspects of this project, which is concerned with a general structural 
description of so-called atonal music, have been preseated in MAC Progress 
.Report m and, more extensively, in MAc-TR-39. During the past year the 
score-reading program, which parses an input string representing standard 
music notation, was improved in the direction of greater generality and ef¬ 
ficiency* and more effective tools were developed to deal with certain basic 
problems oi musical analysis. In particular, the notion of time-point states 
has led to a solution to the problem of segmentation {the determination of 
structural units) and has suggested some useful ways in which relations be¬ 
tween such units might be displayed. The task of constructing an appropriate 
model for this music remains difficult. Work done thus far haa suggested 
several possibilities which are currently being programmed. 
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